Learning Powers of Poisson Binomial Distributions

We introduce the problem of simultaneously learning all powers of a Poisson Binomial Distribution (PBD). A PBD over {1, . . . , n} is the distribution of a sum X = ∑n i=1 Xi, of n independent Bernoulli 0/1 random variables Xi, where E[Xi] = pi. The k’th power of this distribution, for k in a range {1, . . . ,m}, is the distribution of Pk = ∑n i=1 X (k) i , where each Bernoulli random variable X (k) i ∈ {0, 1} has E[X (k) i ] = (pi) . The learning algorithm can query any power Pk several times and succeeds in simultaneously learning all powers in the range, if with probability at least 1− δ: given any k ∈ {1, . . . ,m}, it returns a probability distribution Qk with total variation distance from Pk at most ε. We provide almost matching upper and lower bounds on the query complexity for this problem. We first show an information theoretic lower bound on the query complexity on PBD powers instances with many distinct parameters pi which are significantly separated. This lower bound shows that essentially a constant number of queries is required per each distinct parameter. We almost match this lower bound by examining the query complexity of simultaneously learning all the powers of a special class of PBD’s resembling the PBD’s of our lower bound. We extend the classical minimax risk definition from statistics, dating back to 1930s [Wald 1939], to introduce a framework to study sample complexity of estimating functions of sequences of distributions. Within this framework we show how classic lower bounding techniques, such as Le Cam’s and Fano’s, can be applied to provide sharp lower bounds in our learning model. We study the most fundamental setting of a Binomial distribution, i.e., pi = p, for all i, and provide an optimal algorithm which uses O(1/ε) samples, independent of n,m. Thus, we show how to exploit the additional power of sampling from other powers, that leads to a dramatic increase in efficiency. We also prove a matching lower bound of Ω(1/ε) samples for the Binomial powers problem, by employing our minimax framework. Estimating the parameters of a PBD is known to be hard. Diakonikolas, Kane and Stewart [COLT’16] showed an exponential lower bound of Ω(2) samples to learn the pi’s within error ε. Thus, a natural question is whether sampling from powers of PBDs can reduce this sampling complexity. Using our minimax framework we provide a negative answer to this question, showing that the exponential number of samples is inevitable. We then give a nearly optimal algorithm that learns the pi’s of a PBD using 2 samples from the powers of the PBD, which almost matches our lower bound. The Newton-Girard formulae give relations between the power sums ∑n i=1 z k i , k = 1, . . . , n, of the roots, and the coefficients of a polynomial P (x) = ∏n i=1(x − zi). Thus, if we know the power sums exactly, we can first find the coefficients of P (x) and then compute the roots z1, . . . , zn with an arbitrarily good accuracy. In our problem we only have access to approximate values for the power sums since they correspond to the means of the PBD powers. An intriguing question is to which extent these “noisy” power sum estimations can be used to recover the actual values of p1, . . . , pn within sufficient accuracy. We answer this question by providing close lower and upper bounds on the sample complexity of estimating the parameters of a PBD using samples from its powers.